Refinement of lip shape in sign speech synthesis
نویسنده
چکیده
This paper deals with an analysis of lip shapes during speech that accompanies sign language, referred to as sign speech. A new sign speech database is collected and a new framework for the analysis of mouth patterns is introduced. Using a shape model restricted to the outer lip contour, we show that the articulatory parameters for visual speech alone are not sufficient for representing sign speech. The errors occur mainly for the mouth opening. A correction to the standard articulatory parameters and additional articulatory parameters are investigated to cover the observed mouth patterns and thus refine the synthesised sign speech.
منابع مشابه
Visual analysis of lip coarticulation in VCV utterances
This paper presents an investigation of the visual variation on the bilabial plosive consonant /p/ in three coarticulation contexts. The aim is to provide detailed ensemble analysis to assist coarticulation modelling in visual speech synthesis. The underlying dynamics of labeled visual speech units, represented as lip shape, from symmetric VCV utterances, is investigated. Variation in lip dynam...
متن کاملEvaluation of synthesized sign and visual speech by deaf
This paper is focused on an evaluation of quality of synthesized sign speech and a comparison of sign and visual speech. The evaluation has been performed with the Czech sign speech synthesis system. The system produces a manual component as well as a non-manual component given by the lip articulation. The perception test by deaf children from primary school is scored on the isolated signs. Two...
متن کاملLip animation synthesis: a unified framework for speaking and laughing virtual agent
This paper proposes a unified statistical framework to synthesize speaking and laughing lip animations for virtual agents in real time. Our lip animation synthesis model takes as input the decomposition of a spoken text into phonemes as well as their duration. Our model can be used with synthesized speech. First, Gaussian mixture models (GMMs), called lip shape GMMs, are used to model the relat...
متن کاملVisual analysis of viseme dynamics
Face to face dialogue is the most natural mode of communication between humans. The combination of human visual perception of expression and perception in changes in intonation provides semantic information that communicates idea, feelings and concepts. The realistic modelling of speech movements, through automatic facial animation, and maintaining audio-visual coherence is still a challenge in...
متن کاملModeling visual coarticulation in synthetic talking heads using a lip motion unit inventory with concatenative synthesis
The shape and synchronization of the lip movement with speech seems to be one of the important factors in the acceptability of a synthetic persona, particularly as synthetic beings approach human photo-realism. Most of us cannot lipread nor easily identify a sound by lip-shape alone, but we can readily detect whether the lip movements of a synthetic talking head are acceptable or not. This is t...
متن کامل